Papers with movement pruning paradigm
Block Pruning For Faster Transformers (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pruning methods have proven to be effective at reducing model size, while distillation methods are proven for speeding up inference. |
| Approach: | They propose a block pruning approach that integrates structured pruning methods with the movement pruning paradigm for fine-tuning. |
| Outcome: | The proposed model is 2.4x faster, 74% smaller and faster than distilled models on classification and generation tasks. |